Successive Approximation Coding for Distributed Matrix Multiplication

نویسندگان

چکیده

Coded distributed computing was recently introduced to mitigate the effect of stragglers on systems. This paper combines ideas approximate and coded further accelerate computation. We propose successive approximation coding (SAC) techniques that realize a tradeoff between accuracy speed, allowing system produce approximations increase in over time. If sufficient number compute nodes finish their tasks, SAC exactly recovers desired theoretically provide design guidelines for our techniques, numerically show achieves better accuracy-speed comparison with previous methods.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Analysis of a randomized approximation scheme for matrix multiplication

This note gives a simple analysis of a randomized approximation scheme for matrix multiplication proposed by [Sar06] based on a random rotation followed by uniform column sampling. The result follows from a matrix version of Bernstein’s inequality and a tail inequality for quadratic forms in subgaussian random vectors.

متن کامل

Distributed Adaptive Sampling for Kernel Matrix Approximation

Most kernel-based methods, such as kernel or Gaussian process regression, kernel PCA, ICA, or k-means clustering, do not scale to large datasets, because constructing and storing the kernel matrix Kn requires at least O(n2) time and space for n samples. Recent works [1, 9] show that sampling points with replacement according to their ridge leverage scores (RLS) generates small dictionaries of r...

متن کامل

Straggler Mitigation in Distributed Matrix Multiplication: Fundamental Limits and Optimal Coding

We consider the problem of massive matrix multiplication, which underlies many data analytic applications, in a large-scale distributed system comprising a group of worker nodes. We target the stragglers’ delay performance bottleneck, which is due to the unpredictable latency in waiting for slowest nodes (or stragglers) to finish their tasks. We propose a novel coding strategy, named entangled ...

متن کامل

Communication lower bounds for distributed-memory matrix multiplication

We present lower bounds on the amount of communication that matrixmultiplication algorithms must perform on a distributed-memory parallel computer. We denote the number of processors by P and the dimension of square matrices by n. We show that the most widely-used class of algorithms, the so-called 2-dimensional (2D) algorithms, are optimal, in the sense that in any algorithm that uses only O(n...

متن کامل

Spectrum Approximation Beyond Fast Matrix Multiplication: Algorithms and Hardness

Understanding the singular value spectrum of a matrix A ∈ Rn×n is a fundamental task in countless numerical computation and data analysis applications. In matrix multiplication time, it is possible to perform a full SVD of A and directly compute the singular values σ1, . . . , σn. However, little is known about algorithms that break this runtime barrier. Using tools from stochastic trace estima...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: IEEE journal on selected areas in information theory

سال: 2022

ISSN: ['2641-8770']

DOI: https://doi.org/10.1109/jsait.2022.3190859